The Yi-VL multimodal language model has been launched, including two versions: Yi-VL-34B and Yi-VL-6B. The Yi-VL model demonstrates exceptional capabilities in visual-text understanding and dialogue generation. It has achieved leading performance on English and Chinese datasets. Yi-VL-34B surpasses other multimodal large models with an accuracy rate of 41.6%. The Yi-VL model is based on the LLaVA architecture, featuring strong language understanding and generation abilities.

NExT-GPT is an open source multimodal language model developed by the National University of Singapore, capable of processing text, images, videos, and audio, providing robust support for multimedia AI applications. It features a three-layer architecture, including linear projection, Vicuna LLM core, and modality-specific transformation layers, with intermediate layer training conducted using MosIT technology. The open-source contribution enables researchers and developers to create applications that integrate multimodal inputs, with potential applications spanning a wide range of fields. What sets NExT-GPT apart is its ability to generate modalities based on user requests.

Researchers from Apple and Columbia University have developed the Ferret multimodal language model, aimed at achieving advanced image understanding and description. The Ferret model possesses strong global understanding capabilities, able to handle free text and referenced regions, outperforming traditional models. The researchers created the GRIT dataset, which includes 1.1 million samples, to guide the model in reference and localization tasks. Ferret-Bench evaluations show that Ferret outperforms the best MLLM models by an average of 20.4% while reducing object-related errors.